Moving image coding apparatus and moving image decoding apparatus

ABSTRACT

A macro block size determining unit  1  determines the size of each macro block on a frame-by-frame basis. A macro block dividing unit  2  divides an inputted image into macro blocks each having the size determined by the macro block size determining unit  1 . A macro block coding unit  3  determines a coding mode for each of the macro blocks divided by the macro block dividing unit  2 , and codes pixel values in each of the macro blocks in the determined coding mode.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a division of and claims the benefit of priorityunder 35 U.S.C. § 120 from U.S. application Ser. No. 17/149,872 filedJan. 15, 2021, which a divisional of U.S. application Ser. No.16/505,861 filed Jul. 9, 2019 (now U.S. Pat. No. 11,039,136 issued Jun.15, 2021), which is a divisional of U.S. application Ser. No. 16/131,155filed Sep. 14, 2018 (now U.S. Pat. No. 10,728,550 issued Jul. 28, 2020),which is a division of U.S. application Ser. No. 15/371,516 filed Dec.7, 2016 (now U.S. Pat. No. 10,142,632 issued Nov. 27, 2018), which is adivision of U.S. application Ser. No. 14/667,032 filed Mar. 24, 2015(now U.S. Pat. No. 9,591,308 issued Mar. 7, 2017), which is a divisionof U.S. application Ser. No. 13/918,205 filed Jun. 14, 2013 (now U.S.Pat. No. 9,060,157 issued Jun. 16, 2015), which is a division of U.S.application Ser. No. 12/067,729 filed Mar. 21, 2008 (now U.S. Pat. No.8,467,450 issued Jun. 18, 2013), which is a national stage ofInternational Application No. PCT/JP2006/318861 filed Sep. 22, 2006,which claims the benefit of priority under 35 U.S.C. § 119 from priorJapanese Patent Application No. 2005-278369 filed Sep. 26, 2005, theentire contents of each of which are incorporated herein by reference.

FIELD OF THE INVENTION

The present invention relates to a moving image coding apparatus whichdivides a moving image into rectangular regions and codes the movingimage in units of the rectangular regions, and a moving image decodingapparatus which decodes a coded moving image in units of rectangularregions.

BACKGROUND OF THE INVENTION

As a conventional method of coding or decoding an image signal, therehas been provided MPEG-4 (Moving Picture Experts Group Phase-4) visual(ISO/IEC 14496-2). MPEG-4 is a method of regarding a moving imagesequence as a collection of moving image objects each of which has anarbitrary shape in space and in time, and performing coding and decodingthe moving image sequence in units of each moving image object.

In MPEG-4, each moving image object including a time axis is referred toas a Video Object, each element of a Video Object is referred to as aVideo Object Layer (VOL), each element of a VOL is referred to as aGroup of Video Object Plane (GOV), and image data which shows the stateof a GOV at each time and which is used as a unit to be coded isreferred to as a Video Object Plane (VOP). In a case of handling anobject of rectangular shape, a VOP corresponds to a frame. Each VOP iscoded in units of each macro block which is a unit region to be coded.Each macro block is a rectangular block of 16 pixels×16 lines. Eachmacro block is further divided into blocks of 8 pixels×8 lines, and amotion compensation prediction is performed in units of each macro blockor each block of 8 pixels×8 lines.

Selection of the size of each block for the motion compensationprediction can be changed for each of the macro blocks. Whether to codea target macro block can be determined for each of the macro blocks.When not coding the target macro block (in a non-coding mode), decodeddata about a macro block at the same position (right in back) of animmediately-previous frame are used. When the target macro block iscoded, whether to perform a motion compensation prediction (an intercoding mode) or to perform the coding using only information about thetarget frame without performing any inter frame prediction using amotion compensation prediction (an intra coding mode) can be selected inunits of each macro block.

Furthermore, there has been provided an image coding/decoding method ofbeing able to divide an image into macro blocks having a plurality ofsizes in division of the image into blocks when performing a codingprocess, thereby making a macro block corresponding to a part of theimage with higher importance in its image quality have a smaller sizeand also making a macro block corresponding to a part of the image withlower importance in its image quality have a larger size (for example,refer to patent reference 1).

-   [Patent reference 1] JP,11-146367,A

A problem with such the conventional image coding/decoding method isthat because the macro block size is fixed, the macro block size cannotbe changed adaptively according to the resolution of the image or thedescriptions of the image (the fineness of a pattern of the image, theintensity of a motion in the image, etc.). Particularly, when theresolution of the image becomes high, the use of the fixed macro blocksize adopted by the conventional image coding/decoding method causes acase in which a region covered by macro blocks is localized, andneighboring macro blocks have the same coding mode or the same motionvector. In such a case, because the overhead due to coding modeinformation and motion vector information which are coded in units ofeach macro block increases though the prediction efficiency does notincrease, the whole coding efficiency decreases.

When the image has few motions or any motion in frames of the image isconstant even if the image has a low resolution, the conventional imagecoding/decoding method causes a case in which neighboring macro blockshave the same coding mode (non-coding mode) or the same motion vector.Generally, when the size of each macro block is large, the amount ofinformation about the overhead required in units of each macro block,such as the coding mode, can be reduced. However, because switching ofthe coding mode in units of a unit region having a smaller size than themacro block size cannot be performed, the prediction efficiency in thevicinity of a boundary of a region having large intensity of motion or aregion including different motions (a person and a background, or thelike) decreases, and therefore the coding efficiency decreases. Incontrast, when the size of each macro block is small, although thenumber of macro blocks increases and the amount of information about theoverhead, such as the coding mode, increases, because switching of thecoding mode can be performed adaptively in the vicinity of a boundary ofa region having large intensity of motion or a region includingdifferent motions (a person and a background, or the like), thepredictive efficiency is improved and therefore the coding efficiency isimproved.

Furthermore, a problem with the technology disclosed by above-mentionedconventional patent reference 1 is that because the size is changedadaptively within each frame, the amount of codes of the rectangularsize information is large.

The present invention is made in order to solve the above-mentionedproblems, and it is therefore an object of the present invention toprovide a moving image coding apparatus which can code an image whilechanging the size of each of rectangular regions adaptively according tothe resolution or the descriptions of the image, and which can suppressthe increase in the rectangle size information of the coded image due tothe changing, and a moving image decoding apparatus.

DISCLOSURE OF THE INVENTION

A moving image coding apparatus in accordance with the present inventionincludes: a rectangular region size determination means for determininga size of each of the rectangular regions for either each frame or eachsequence which is a set of a plurality of continuous frames; arectangular region dividing means for dividing an inputted image intorectangular regions each having the size determined by the rectangularregion size determination means; and a rectangular region coding meansfor determining a coding mode for each of the rectangular regions, andfor coding pixel values in each of the rectangular regions in thedetermined coding mode.

Because the moving image coding apparatus in accordance with the presentinvention determines the size of each of the rectangular regions foreach frame or each sequence which is a set of a plurality of continuousframes, the moving image coding apparatus can code the image whilechanging the rectangular region size adaptively, and can suppress theincrease in the rectangular size information of the coded image due tothis changing.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a block diagram showing a moving image coding apparatus inaccordance with Embodiment 1 of the present invention;

FIG. 2 is an explanatory drawing showing an example (a first example) ofdivision of an image into regions each of which is used as a unit formotion compensation in the moving image coding apparatus in accordancewith Embodiment 1 of the present invention;

FIG. 3 is an explanatory drawing showing an example (a second example)of division of an image into regions each of which is used as a unit formotion compensation in the moving image coding apparatus in accordancewith Embodiment 1 of the present invention;

FIG. 4 is an explanatory drawing of motion vector detection in themoving image coding apparatus in accordance with Embodiment 1 of thepresent invention;

FIG. 5 is an explanatory drawing of intra prediction in the moving imagecoding apparatus in accordance with Embodiment 1 of the presentinvention;

FIG. 6 is an explanatory drawing of a case in which macro block sizeinformation is changed in units of slices in the moving image codingapparatus in accordance with Embodiment 1 of the present invention; and

FIG. 7 is a block diagram showing a moving image decoding apparatus inaccordance with Embodiment 2 of the present invention.

PREFERRED EMBODIMENTS OF THE INVENTION

Hereafter, in order to explain this invention in greater detail, thepreferred embodiments of the present invention will be described withreference to the accompanying drawings.

Embodiment 1

FIG. 1 is a block diagram showing a moving image coding apparatus inaccordance with Embodiment 1 of the present invention.

In the figure, the moving image coding apparatus is provided with amacro block size determining unit (a rectangular region size determiningmeans) 1, a macro block dividing unit (a rectangular region dividingmeans) 2, a macro block coding unit (a rectangular region coding means)3, and an entropy coding unit (a rectangular region size informationmultiplexing means) 4. The macro block coding unit 3 is provided with acoding mode determining part (a coding mode determining means) 5, amotion compensation region dividing part (a motion compensation regiondividing means) 6, a motion detecting part (a motion detecting means) 7,a motion compensation part (a motion compensation means) 8, a subtractor9, a block dividing part (a block dividing means) 10, atransformation/quantization part (a transformation/quantization means)11, an inverse quantization/inverse transformation part 12, an adder 13,a reference image memory 14, an intra prediction block dividing part (anintra prediction block dividing means) 15, and an intra prediction part(an intra prediction means) 16.

The macro block size determining unit 1 is a functional unit fordetermining the size of each of macro blocks on a frame-by-frame basis,and outputs this size as macro block size specification information(rectangular region size information) 102. The macro block dividing unit2 is a functional unit for dividing an inputted image 101 into macroblocks each having the size determined by the macro block sizedetermining unit 1. The macro block coding unit 3 has a function ofdetermining a coding mode for each macro block, and coding pixel valuesin each macro block in the determined coding mode.

The entropy coding unit 4 has a function of multiplexing the macro blocksize specification information 102 determined by the macro block sizedetermining unit 1, block size specification information 108, and thecoding mode information 106 outputted from the coding mode determiningpart 5, coefficient data 109 acquired by the transformation/quantizationpart 11, and motion vector information 107 acquired by the motiondetecting part 7 into a bit stream 103.

The coding mode determining part 5 is a functional unit for determiningthe coding mode for each macro block. The motion compensation regiondividing part 6 is a functional unit for dividing the interior of eachmacro block into regions determined by the coding mode determining part5. The motion detecting part 7 is a functional unit for performing amotion detection in units of each of the regions divided by the motioncompensation region dividing part 6. The motion compensation part 8 is afunctional unit for performing a motion compensation prediction usingthe motion vector information 107 acquired by the motion detecting part7 so as to acquire a prediction image. The subtractor 9 is the one foracquiring differences between the pixel values of the prediction imageacquired by the motion compensation part 8 and those of the inputtedimage. The block dividing part 10 is a functional unit for acquiring thepixel value differences acquired by the subtractor 9 as a predictionerror image in units of each macro block, and for dividing the interiorof each macro block of the inputted image or the prediction error imageinto rectangular blocks each having a size, which is determinedbeforehand, on the basis of the block size specification information108.

The transformation/quantization part 11 is a functional unit forcarrying out an orthogonal transformation process and quantization inunits of each of the blocks divided by the block dividing part 10, andfor outputting the coefficient data 109. The inversequantization/inverse transformation part 12 is a functional unit forcarrying out inverse quantization/inverse transformation on the basis ofthe coefficient data 109 outputted from the transformation/quantizationpart 11 so as to acquire a decoded prediction error image. The adder 13is the one for adding the decoded prediction error image acquired by theinverse quantization/inverse transformation part 12 and the predictionimage acquired by the motion compensation part 8 so as to acquire adecoded image. The reference image memory 14 is the one for storing thedecoded image outputted from the adder 13 as a reference image.

Next, the operation of the moving image coding apparatus of Embodiment 1will be explained.

First, an inputted image 101 is inputted to the macro block dividingunit 2 and is divided into macro blocks each having a predeterminedmacro block size on the basis of the macro block size specificationinformation 102 which is determined on a frame-by-frame basis. The macroblock size specification information 102 is the one showing a size whichcan be selected from among, for example, 16 pixels×16 lines and 32pixels×32 lines, and will be explained below. The macro block sizespecification information 102 specified is inputted to the entropycoding unit 4 and is entropy-coded, and is then multiplexed into a bitstream 103.

The macro block size specification information 102 is outputted from themacro block size determining unit 1. The macro block size determiningunit 1 calculates a difference between frames, and, when this framedifference is small, uses macro blocks of 32 pixels×32 lines, whereaswhen the frame difference is large, uses macro blocks of 16 pixels×16lines. As a second method, the macro block size determining unitperforms a motion detection in units of 32 pixels×32 lines, and alsoperforms a motion detection in units of 16 pixels×16 lines, andcalculates a prediction error for each size, and uses one of the sizeswhich provides a smaller prediction error as the macro block size. As athird method, the macro block size determining unit calculates avariance of pixel values in units of 32 pixels×32 lines and a varianceof pixel values in units of 16 pixels×16 lines, and uses one of thesizes which provides a smaller averaged variance in a frame as the macroblock size. As an alternative, using a combination of two or more ofthese methods, the macro block size can be determined.

As a fourth method, the macro block size determining unit canautomatically determine the macro block size according to the resolutionof the image by, for example, using, in a case in which the inputtedimage has a resolution higher than a predetermined resolution, the sizeof 32 pixels×32 lines, or using the size of 16 pixels×16 linesotherwise. In this case, because the macro block size is decided by theresolution, it is not necessary to multiplex the macro block sizespecification information 102 into the bit stream 103.

As an alternative, some possible values of the macro block size can bepredetermined respectively according to pieces of information (referredto as profiles) each of which is a predetermined combination of codingparameters which are available for a usage pattern, and the macro blocksize can be automatically determined from the possible values accordingto a selected profile.

The inputted image 101 which are divided into macro blocks is inputtedto the motion compensation region dividing part 6, the intra predictionblock dividing part 15, or the block dividing part 10 within the macroblock coding unit 3 according to the coding mode determined for eachmacro block.

Macro block type information, motion compensation region shapeinformation 104, and intra prediction block size information 105 areincluded in the coding mode determined for each macro block. The macroblock type information is the one which can be selected from informationshowing a mode (an inter mode) in which inter-frame prediction coding iscarried out, information showing a mode (an intra mode) in whichintra-frame coding is carried out, and information showing a mode (anintra prediction mode) in which intra-frame prediction coding is carriedout. Although the selection of a mode from these modes is carried out bythe coding mode determining part 5, because the selection method ofselecting a mode from these modes is not related directly to thefeatures of the present invention, the explanation of the method will beomitted hereafter.

The coding mode determining part 5 determines a further-detailed codingmode for each of the above-mentioned coding modes. More specifically, inthe case of the inter mode, the coding mode determining part determinesmotion compensation region shape information 104 showing units on eachof which motion compensation is to be performed, and, in the case of theintra prediction mode, determines intra prediction block sizeinformation 105 showing units on each of which intra prediction is to beperformed. The description of each of the pieces of information will beexplained later. The various coding modes determined by the coding modedetermining part 5 are given to the entropy coding unit 4 as coding modeinformation 106, and are coded in units of each macro block by theentropy coding unit 4 and are multiplexed into the bit stream.

First, a case in which the inter mode is selected will be explained.

The inputted image divided into macro blocks by the macro block dividingunit 2 is inputted to the motion compensation region dividing part 6,and is further divided into regions each having a certain shape, whichare units on each of which motion compensation is to be performed, onthe basis of the motion compensation region shape information 104selected by the coding mode determining part 5.

An example of the division of each macro block into regions each ofwhich is used as a unit for motion compensation is shown in FIGS. 2 and3 .

In the case in which the macro block size is 16 pixels×16 lines, eachmacro block is divided into either of regions shown in (a) to (d) ofFIG. 2 (a region of 16 pixels×16 lines, regions of 8 pixels×16 lines,regions of 16 pixels×8 lines, or regions of 8 pixels×8 lines), and, in acase in which each macro block is divided into regions shown in (d),each block of 8 pixels×8 lines can be further divide into either ofregions shown in (e) to (h) of FIG. 2 (a region of 8 pixels×8 lines,regions of 4 pixels×8 lines, regions of 8 pixels×4 lines, or regions of4 pixels×4 lines). In the case in which the macro block size is 32pixels×32 lines, each macro block can be divided into either of regionsshown in (a) to (d) of FIG. 2 , as in the case in which the macro blocksize is 16 pixels×16 lines. As another example, each macro block can bedivided into regions shown in either of (j) to (k) of FIG. 3 (a regionof 32 pixels×32 lines, regions of 16 pixels×32 lines, regions of 32pixels×16 lines, or regions of 16 pixels×16 lines) and, in a case inwhich each macro block is divided into regions shown in (k), each blockof 16 pixels×16 lines can be further divided into regions shown ineither of (j) to (k) of FIG. 3 . Information specifying one of thesedividing methods is the motion compensation region shape information104. As mentioned above, in the motion compensation region shapeinformation 104, the region shapes which can be selected can be changedaccording to the macro block size.

The inputted image in which each macro block is divided into regionshaving a certain shape, each of which is used as a unit on which motioncompensation is to be performed, is inputted to the motion detectingpart 7, and a motion detection is performed on the inputted image usingone or more reference images stored in the reference image memory 14 bythe motion detecting part. The motion detecting part 7 searches for apredetermined region of a reference image in units of each of theregions determined by the motion compensation region dividing part 6 soas to detect an optimal motion vector. FIG. 4 is an explanatory drawingof this motion vector detection. Because the motion vector detectingoperation is known, the detailed explanation of the operation will beomitted hereafter.

The motion vector information 107 acquired by the motion detecting part7 is inputted to the entropy coding unit 4, is entropy-coded andmultiplexed into the bit stream 103, and is also inputted to the motioncompensation part 8. The motion compensation part 8 generates aprediction image using a reference image stored in the reference imagememory 14 and the motion vector information 107 from the motiondetecting part 7. A difference between the prediction image acquired bythe motion compensation part 8 and the inputted image which are dividedinto macro blocks is acquired by the subtractor 9, and this differenceis inputted to the block dividing part 10 as a prediction error image.The block dividing part 10 divides this prediction error image intoblocks each of which is used as a unit on which transformation(orthogonal transformation like DCT) and quantization are to beperformed on the basis of the block size specification information 108.

The block size specification information 108 is the one which can beselected from, for example, information showing 8 pixels×8 lines andinformation showing 4 pixels×4 lines. As another example, the selectableblock size specification information 108 can be changed according to themacro block size. For example, in the case in which each macro block hasa size of 16 pixels×16 lines, the block size specification information108 is the one which can be selected from information showing 8 pixels×8lines and information showing 4 pixels×4 lines, whereas in the case inwhich each macro block has a size of 32 pixels×32 lines, the block sizespecification information 108 is the one which can be selected frominformation showing 16 pixels×16 lines and information showing 8pixels×8 lines. The selected block size specification information 108 isinputted to the entropy coding unit 4, is entropy-coded, and ismultiplexed into the bit stream 103.

As another example, the block size can be determined according to themacro block size. For example, in the case in which each macro block hasa size of 16 pixels×16 lines, the block size is set to 4 pixels×4 lines,and, in the case in which each macro block has a size of 32 pixels×32lines, the block size is set to 8 pixels×8 lines. In this case, becausethe block size is determined according to the macro block size, it isnot necessary to multiplex the block size specification information intothe bit stream 103.

The prediction error image divided into blocks is inputted to thetransformation/quantization part 11, and orthogonal transformation andquantization are performed on the prediction error image. Coefficientdata 109 acquired as the results of the orthogonal transformation andquantization processing are entropy-coded by the entropy coding unit 4,and are multiplexed into the bit stream 103. The coefficient data 109acquired by the transformation/quantization part 11 are further inputtedto the inverse quantization/inverse transformation part 12, and inversequantization/inverse transformation is performed on the coefficientdata, so that a decoded prediction error image is acquired. The decodedprediction error image is added to the prediction image acquired by themotion compensation part 8 by the adder 13, and the addition result isthen outputted as a decoded image. Because the decoded image is used asa reference image in performing coding of subsequent inputted images,the decoded image is stored in the reference image memory 14.

Next, a case in which the intra mode is selected will be explained.

The inputted image which is divided into macro blocks by the macro blockdividing unit 2 is inputted to the block dividing part 10. The operationof the block dividing part 10 is the same as that in the case of theinter mode. Orthogonal transformation and quantization are performed onthe inputted image divided into blocks by thetransformation/quantization part 11. Coefficient data 109 acquired asthe results of the orthogonal transformation and quantization processingare entropy-coded by the entropy coding unit 4, and are multiplexed intothe bit stream 103. The coefficient data 109 acquired by thetransformation/quantization part 11 are further inputted to the inversequantization/inverse transformation part 12, and inversequantization/inverse transformation is performed on the coefficientdata, so that a decoded image is acquired. Because the decoded image isused as a reference image in performing coding of subsequent inputtedimages, the decoded image is stored in the reference image memory 14.

Next, a case in which the intra prediction mode is selected will beexplained.

The inputted image which is divided into macro blocks by the macro blockdividing unit 2 is inputted to the intra prediction block dividing part15. The intra prediction block dividing part 15 divides each macro blockinto blocks each of which is used as a unit on which an intra predictionis to be performed on the basis of the intra prediction block sizeinformation 105. The intra prediction block size information 105 is theone which can be selected from information showing 16 pixels×16 linesand information showing 4 pixels×4 lines. As another example, theselectable intra prediction block size information 105 can be changedaccording to the macro block size. For example, in the case in whicheach macro block has a size of 16 pixels×16 lines, the intra predictionblock size information 105 is the one which can be selected frominformation showing 16 pixels×16 lines and information showing 4pixels×4 lines, and in the case in which each macro block has a size of32 pixels×32 lines, the intra prediction block size information 105 isthe one which can be selected from information showing 32 pixels×32lines and information showing 8 pixels×8 lines.

The inputted image which is divided into blocks by the intra predictionblock dividing part 15 is inputted to the intra prediction part 16, andan intra prediction is performed on the inputted image by the intraprediction part. Although a detailed explanation of the intra predictionwill be omitted hereafter because the intra prediction is known, theintra prediction part predicts each pixel value in each target block tobe coded from the pixel values of coded neighboring blocks (from thepixel values of blocks A to D of FIG. 5 ) to generate a predictionimage, as shown in FIG. 5 . A difference between the prediction imageacquired by the intra prediction part 16 and the inputted image dividedinto macro blocks is acquired by the subtractor 9, and this differenceis inputted to the block dividing part 10 as a prediction error image.The operation of the block dividing part 10 is the same as that in thecase of the inter mode.

The prediction error image divided into blocks is inputted to thetransformation/quantization part 11, and orthogonal transformation andquantization are performed on the prediction error image by thetransformation/quantization part. Coefficient data 109 acquired as theresults of the orthogonal transformation and quantization processing areentropy-coded by the entropy coding unit 4, and are multiplexed into thebit stream 103. The coefficient data 109 acquired by thetransformation/quantization part 11 are further inputted to the inversequantization/inverse transformation part 12, and inversequantization/inverse transformation is performed on the coefficient databy the inverse quantization/inverse transformation part, so that adecoded prediction error image is acquired. The decoded prediction errorimage is added to the prediction image acquired by the intra predictionpart 16 by the adder 13, and the addition result is outputted as adecoded image. Because the decoded image is used as a reference image inperforming coding of subsequent inputted images, the decoded image isstored in the reference image memory 14.

In the above-mentioned embodiment, the macro block size information canbe changed on a frame-by-frame basis. As shown in an example shown inFIG. 6 , the macro block size information can be changed in units ofeach set (referred to as each slice) of some of a plurality of macroblocks into which the interior of each frame is divided. The codingprocessing has only to be carried out, as mentioned in the aboveembodiment, by assuming each slice as one frame. At this time, the macroblock size specification information 102 is multiplexed into the bitstream 103 in units of each slice.

The macro block size information can be changed in units of eachsequence which is a set of a plurality of continuous frames. At thistime, the macro block size specification information 102 is multiplexedinto the bit stream 103 in units of each sequence.

As a variant, whether to change the macro block size specificationinformation for each frame or for each sequence can be selected. In thiscase, information indicating whether to change the macro block sizespecification information for each frame is multiplexed into the bitstream 103 in units of each sequence, and, when changing of the macroblock size specification information for each frame is selected, themacro block size specification information 102 is multiplexed into thebit stream 103 in units of each frame, whereas when changing of themacro block size specification information for each sequence isselected, the macro block size specification information 102 ismultiplexed into the bit stream 103 in units of each sequence.

As mentioned above, the moving image coding apparatus in accordance withEmbodiment 1, which divides a moving image into rectangular regions andcodes each of the rectangular regions independently, includes: therectangular region size determination means for determining a size ofeach of the rectangular regions for either each frame or each sequencewhich is a set of a plurality of continuous frames; the rectangularregion dividing means for dividing an inputted image into rectangularregions each having the size determined by the rectangular region sizedetermination means; and the rectangular region coding means fordetermining a coding mode for each of the rectangular regions, and forcoding pixel values in each of the rectangular regions in the determinedcoding mode.

Therefore, the moving image coding apparatus can select an appropriaterectangular region size according to the descriptions of the image(e.g., the intensity of a motion in the image, the fineness of a patternin the image, etc.), the resolution of the image, or a profile, therebybeing able to improve the coding efficiency, and can reduce the increasein the amount of codes due to the addition of the rectangle sizeinformation because, for example, only several bits of code amount foreach frame or each sequence is needed as the code amount of therectangle size information.

In addition, because the moving image coding apparatus in accordancewith Embodiment 1 includes the rectangular region size informationmultiplexing means for multiplexing the rectangular region sizeinformation determined by the rectangular region size determinationmeans into a bit stream, the moving image coding apparatus can changethe rectangular region size adaptively according to the characteristicsof the image, or the like.

Furthermore, in the moving image coding apparatus in accordance withEmbodiment 1, because the rectangular region size determination meansdetermines the rectangular region size according to the resolution ofthe inputted image. Therefore, the moving image coding apparatus doesnot need to multiplex the rectangular region size information into thebit stream, and can reduce the overhead of the processing.

In addition, in the moving image coding apparatus in accordance withEmbodiment 1, the rectangular region coding means includes: the codingmode determining means for determining the coding mode for each of therectangular regions; the motion compensation region dividing means fordividing an interior of each of the rectangular regions into regionsdetermined by the coding mode determining means; the motion detectingmeans for performing a motion detection on each of the regions dividedby the motion compensation region dividing means; the motioncompensation means for performing a motion compensation prediction usingmotion vector information acquired by the motion detecting means so asto acquire a prediction image; the block dividing means for acquiringdifferences between pixel values of the prediction image and pixelvalues in each of the rectangular regions as a prediction error image inunits of each of the rectangular regions, and for dividing an interiorof each of the rectangular regions of either the inputted image or theprediction error image into rectangular blocks each having apredetermined size; and the transformation/quantization means forperforming an orthogonal transformation and quantization process on eachof the blocks divided by the block dividing means. Therefore, the movingimage coding apparatus can also support the inter mode in which itperforms inter-frame prediction coding as the coding mode determined foreach rectangular region size.

In the moving image coding apparatus in accordance with Embodiment 1,the rectangular region coding means includes: the intra prediction blockdividing means for dividing the interior of each of the rectangularregions into rectangular blocks defined by the coding mode; the intraprediction means for performing an intra prediction on each of therectangular blocks divided by the intra prediction block dividing meansso as to acquire a prediction image; the block dividing means foracquiring a prediction error image in units of each of the rectangularregions as the differences between the pixel values of the predictionimage and the pixel values in each of the rectangular regions, and fordividing the interior of each of the rectangular regions of either theinputted image or the prediction error image into rectangular blockseach having the predetermined size; and the transformation/quantizationmeans for performing a quantization and orthogonal transformationprocess on each of the blocks divided by the block dividing means.Therefore, the moving image coding apparatus can also support the intraprediction mode in which it performs intra-frame prediction coding asthe coding mode determined for each rectangular region size.

Embodiment 2

Embodiment 2 relates to a moving image decoding apparatus which decodesa bit stream which is coded with the macro block size being selectedindependently on a frame-by-frame basis.

FIG. 7 is a block diagram of the moving image decoding apparatus inaccordance with Embodiment 2.

In the figure, the moving image decoding apparatus is provided with anentropy decoding unit 21 and a macro block decoding unit (a rectangularregion decoding means) 22. In this case, the entropy decoding unit 21 isa processing unit which performs entropy decoding on an inputted bitstream 201, and which implements a rectangular region size determinationmeans for determining the size of each of rectangular regions in unitsof each frame or in units of each sequence which is a set of a pluralityof continuous frames, a coding mode decoding means for decoding a codingmode which is determined for each rectangular region, amotion-vector-information decoding means for decoding motion vectorinformation in units of each region for motion compensation, and a blocksize determining means for determining the size of each of rectangularblocks which is used as a unit on which inverse quantization and inverseorthogonal transformation are to be performed.

The macro block decoding unit 22 is comprised of a coding mode changingpart 23, a motion compensation region dividing part (a motioncompensation region dividing means) 24, a motion compensation part (amotion compensation means) 25, a reference image memory 26, an inversequantization/inverse transformation part (an inversequantization/inverse orthogonal transformation means) 27, an adder 28,an intra prediction block dividing part 29, and an intra prediction part(an intra prediction means) 30.

The coding mode changing part 23 is a functional unit for changing thecoding mode on the basis of macro block type information 204. The motioncompensation region dividing part 24 is a functional unit for dividingthe interior of each macro block into regions which are defined bymotion compensation region shape information 206. The motioncompensation part 25 is a functional unit for performing a motioncompensation prediction using motion vector information 207 so as toacquire a prediction image. The inverse quantization/inversetransformation part 27 is a functional unit for carrying out an inversequantization/inverse orthogonal transformation process on coefficientdata 208 in units of each macro block on the basis of block sizespecification information 203 so as to acquire a prediction error image.The adder 28 is the one for adding the prediction error image outputtedfrom the inverse quantization/inverse transformation part 27 and theprediction image outputted from the motion compensation part 25 so as toacquire a decoded image 209. The reference image memory 26 is the onefor storing the decoded image 209 as a reference image.

Next, the operation of the moving image decoding apparatus in accordancewith Embodiment 2 will be explained.

First, an input bit stream 201 is inputted to the entropy decoding unit21, and macro block size specification information 202, block sizespecification information 203, coding mode (macro block type information204, intra prediction block size information 205, and motioncompensation region shape information 206), motion vector information207, and coefficient data 208 are decoded. The macro block sizespecification information 202 is the information decoded in units ofeach frame, and the coding mode is the information decoded in units ofeach macro block having a size determined by the macro block sizespecification information 202. The motion vector information 207 is theinformation decoded in units of the decoded motion compensation regionshape information 206.

Next, the macro block size is determined by using the macro block sizespecification information 202, and subsequent decodings are carried outin units of each macro block having the determined size by the macroblock decoding unit 22. In a case in which the macro block type is theinter mode, a motion compensation prediction is carried out, whereas ina case in which the macro block type is the intra prediction mode, anintra prediction is carried out.

First, the case in which the macro block type is the inter mode will beexplained.

The motion compensation region dividing part 24 divides the interior ofeach macro block having a specified size into regions each of which isused as a unit on which motion compensation is to be performed on thebasis of the motion compensation region shape information 206 decoded bythe entropy decoding unit 21. The descriptions of the motioncompensation region shape information 206 are the same as that of themotion compensation region shape information 104 shown in Embodiment 1.The motion compensation part 25 generates a prediction image in units ofeach region determined by the motion compensation region dividing part24 by using the decoded motion vector information 207 and a referenceimage included in the reference image memory 26. The motion vectorinformation 207 is also the same as that of the motion vectorinformation 107 shown in Embodiment 1.

The coefficient data 208 decoded by the entropy decoding unit 21 areinputted to the inverse quantization/inverse transformation part 27. Theinverse quantization/inverse transformation part 27 performs an inversequantization/inverse transformation process on the coefficient data onthe basis of the decoded block size specification information 203. Morespecifically, when the block size specification information 203 shows,for example, 4 pixels×4 lines, the inverse quantization/inversetransformation part carries out the inverse quantization/inversetransformation in units of 4 pixels×4 lines, so that a prediction errorimage is decoded.

The prediction image generated by the motion compensation part 25 andthe prediction error image decoded by the inverse quantization/inversetransformation part 27 are added by the adder 28, and a decoded image209 is acquired. Because the decoded image 209 is used for subsequentdecodings, it is stored in the reference image memory 26.

Next, the case in which the coding mode is the intra mode will beexplained.

The decoded coefficient data 208 are inputted to the inversequantization/inverse transformation part 27. The operation of theinverse quantization/inverse transformation part 27 is the same as thatin the case of the inter mode. The result of the inversequantization/inverse transformation is outputted as a decoded image 209.The decoded image 209 is stored in the reference image memory 26, as inthe case of the inter mode.

Next, the case in which the coding mode is the intra prediction modewill be explained.

The intra prediction block dividing part 29 divides the interior of eachmacro block into blocks each of which is used as a unit on which anintra prediction is to be performed on the basis of the intra predictionblock size information 205 decoded by the entropy decoding unit 21. Thedescription of the intra prediction block size information 205 is thesame as that of the intra prediction block size information 105 shown inEmbodiment 1. The intra prediction part 30 carries out an intraprediction in units of each block determined by the intra predictionblock dividing part 29. Although a detailed explanation of the intraprediction will be omitted hereafter because the intra prediction isknown, the intra prediction part generates predicted values of a targetblock to be coded (a prediction image) using the pixel values ofalready-decoded neighboring blocks, like the intra prediction part 16 ofEmbodiment 1.

On the other hand, the coefficient data 208 decoded by the entropydecoding unit 21 are inputted to the inverse quantization/inversetransformation part 27. The operation of inverse quantization/inversetransformation part 27 is the same as that in the case of the intermode. The result of the inverse quantization/inverse transformation isoutputted as a prediction error image. The prediction image generated bythe intra prediction part 30 and the prediction error image decoded bythe inverse quantization/inverse transformation part 27 are added by theadder 28, and a decoded image 209 is acquired. Because the decoded image209 is used for subsequent decodings, it is stored in the referenceimage memory 26.

In above-mentioned Embodiment 2, the macro block size specificationinformation is decoded in units of each frame, and the inputted bitstream is decoded with the macro block size being changed for eachframe. As an alternative, the macro block size specification informationcan be decoded in units of each slice which is a set of a plurality ofmacro blocks, and the inputted bit stream can be decoded with the macroblock size being changed for each slice.

In another example, the macro block size specification information canbe decoded in units of each sequence which is a set of a plurality offrames, and the inputted bit stream can be decoded with the macro blocksize being changed for each sequence.

In another example, information indicating whether to change the macroblock size for each frame or for each sequence is decoded in units ofeach sequence, and, when changing of the macro block size for each frameis selected, the macro block size specification information is decodedin units of each frame, whereas when changing of the macro block sizefor each sequence is selected, the macro block size specificationinformation is decoded in units of each sequence.

Furthermore, the entropy decoding unit 21 decodes the macro block sizespecification information multiplexed into the bit stream, as mentionedabove. As an alternative, the moving image decoding can automaticallydetermine the macro block size specification information according tothe resolution of the image to be decoded by using, for example, 32pixels×32 lines in a case in which the image to be decoded has aresolution higher than a predetermined resolution, or 16 pixels×16 linesin a case in which the image to be decoded has a resolution lower thanthe predetermined resolution.

As mentioned above, the moving image decoding apparatus in accordancewith Embodiment 2, which divides a moving image into rectangular regionsand codes each of the rectangular regions independently, includes therectangular region size determination means for determining the size ofeach of the rectangular regions for either each frame or each sequencewhich is a set of a plurality of continuous frames; the coding modedecoding means for decoding a coding mode determined for the each of therectangular regions; and the rectangular region decoding means fordecoding pixel values in each of the rectangular regions in the codingmode decoded by the coding mode decoding means. Therefore, the movingimage decoding apparatus can decode a moving image which is codedefficiently with the rectangular region size being changed on aframe-by-frame or sequence-by-sequence basis. Furthermore, because therectangular region size is changed on a frame-by-frame orsequence-by-sequence basis, the present embodiment offers an advantageof reducing the workload required for the changing of the rectangularregion size, as compared with, for example, a case in which therectangular region size is changed within each frame.

Furthermore, in the moving image decoding apparatus in accordance withEmbodiment 2, because the rectangular region size determination meansdecodes rectangular region size information which is multiplexed into abit stream in units of each frame or each sequence which is a set of aplurality of continuous frames, and determines the rectangular regionsize on the basis of the decoded result, the moving image decodingapparatus can change the rectangular region size easily and adaptivelyaccording to the characteristics of the image, or the like.

In addition, in the moving image decoding apparatus in accordance withEmbodiment 2, because the rectangular region size determination meansdetermines the rectangular region size according to the resolution of animage to be decoded, the moving image decoding apparatus does not needto decode the rectangular region size information separately, and canreduce the overhead of the processing.

Furthermore, in the moving image decoding apparatus in accordance withEmbodiment 2, the coding mode decoded by the coding mode decoding meansincludes motion compensation region shape information showing a shape ofeach of unit regions for motion compensation, and the rectangular regiondecoding means includes: the motion compensation region dividing meansfor dividing an interior of each of the rectangular regions into regionsdefined by the motion compensation region shape information, amotion-vector-information decoding means for decoding motion vectorinformation in units of each of the unit regions for motioncompensation; the motion compensation means for performing a motioncompensation prediction using the motion vector information acquired bythe motion vector decoding means so as to acquire a prediction image;the block size determining means for determining a size of each ofrectangular blocks which is used as a unit on which inverse quantizationand inverse orthogonal transformation are to be performed; and theinverse quantization/inverse orthogonal transformation means forperforming an inverse quantization/inverse orthogonal transformationprocess on the each of the rectangular blocks. Therefore, the movingimage decoding apparatus can also support, as a coding mode, the intermode in which it performs inter-frame prediction coding.

In addition, in the moving image decoding apparatus in accordance withEmbodiment 2, because the coding mode decoded by the coding modedecoding means includes intra prediction block size information showinga shape of each of unit regions for intra prediction, and therectangular region decoding means includes the intra prediction meansfor performing an intra prediction on each of the rectangular blocksdefined by the intra prediction block size information so as to acquirea prediction image, the moving image decoding apparatus can alsosupport, as a coding mode, the intra prediction mode in which itperforms intra-frame prediction coding.

Furthermore, in the moving image decoding apparatus in accordance withEmbodiment 2, because the motion compensation region shape informationis the decoded shape of each of the unit regions for motion compensationwhich is set up for each rectangular region size, the moving imagedecoding apparatus can also support, as a coding mode determined foreach rectangular region size, the inter mode in which it performsinter-frame prediction coding.

In addition, in the moving image decoding apparatus in accordance withEmbodiment 2, because the intra prediction block size information is adecoded intra prediction block size which is set up for each rectangularregion size, the moving image decoding apparatus can also support, as acoding mode determined for each rectangular region size, the intraprediction mode in which it performs intra-frame prediction coding.

INDUSTRIAL APPLICABILITY

As mentioned above, the moving image coding apparatus in accordance withthe present invention which can code an image while changing the size ofeach of rectangular regions adaptively according to the resolution orthe descriptions of the image (in the vicinity of a boundary of a regionhaving large intensity of motion or a region including different motions(a person and a background, or the like), and which can suppress theincrease in the rectangle size information of the coded image due to thechanging, and the moving image decoding apparatus in accordance with thepresent invention are suitable for applications in apparatus which cantransmit and receive a digital moving image (e.g., a mobile terminalwith a TV phone function).

1. (canceled)
 2. A moving image decoder which divides an encoded bitstream, the encoded bit stream obtained by dividing a moving image intoblocks, the moving image decoder comprising: a block size decoderdecoding information indicating whether to change block size informationfor a frame or for a sequence; a block size determiner determining thesize of the blocks according to the block size information and theinformation; a coding mode decoder decoding a coding mode determined foreach of the blocks; a block decoder decoding pixel values in each of theblocks using the coding mode; and a memory storing the decoded pixelvalues for outputting the moving image.